Complex graphics usually consist of geometric primitives. Based on the recognition of simple geometric primitives, complex graphics recognition focuses on the spatial relationship between the graphic elements. Geometric structure is too complex to use the heuristic rules, and the existing geometric structure analysis methods are also too complicated to use the traditional method. The core technical issues of the structural analysis for the hand-drawn geometry recognition was analyzed, a geometric structure description model based on Scalable Vector Graphics (SVG) tag was designed to represent the graphics by the formal description of the graphical elements and their constraints. It used SVG tags to store the graphical elements and their constraints, and identified the geometry shape and its internal relationship by parsing the SVG tags. It was validated in a prototype system, namely GeoSketch, with promising effect. The experimental results show that the proposed method is simple and low-dimensional, it is easy for determing the shape of the GeoSketch and its internal relations.